ATOM Documentation

← Back to App

Comprehensive Test Coverage Roadmap

**Status**: PLANNED FOR FUTURE SPRINT

**Priority**: HIGH (Production Readiness)

**Estimated Effort**: 2-3 weeks (80-120 hours)

**Target**: 80% test coverage baseline

---

Executive Summary

ATOM SaaS platform currently has **~20% combined test coverage** (19% frontend, 12% backend). This document provides a comprehensive roadmap to achieve **80% coverage** through a dedicated 2-3 week testing sprint.

**Current Coverage (2026-02-22)**:

  • Frontend: 19.01% (3,606 tests, 24.61s execution)
  • Backend: 12.16% (2,786 tests, 3:03 execution, 5,803/66,072 statements)
  • **Combined: ~20%**

**Target Coverage**:

  • Frontend: 80%
  • Backend: 80%
  • Critical modules (brain systems, governance): 90%

**Gap**: 60 percentage points

---

Phase 1: Fix Collection Errors & Infrastructure (Week 1, Days 1-2)

Goal

Unblock existing tests and establish accurate coverage measurement.

Tasks

1.1 Fix Backend Collection Errors (8-12 hours)

**Priority**: CRITICAL - Blocks other tests

**Files with Errors** (17 total):

  1. tests/integration/test_hosting_integration.py
  2. tests/integration/test_package_performance_integration.py
  3. tests/integration/test_package_security_integration.py
  4. tests/integration/test_public_marketplace_api.py
  5. tests/unit/test_atom_agent_endpoints.py
  6. tests/unit/test_auth_core.py - May work individually
  7. tests/unit/test_coverage_analyzer.py - Syntax error at line 141
  8. tests/unit/test_deployment_agent.py
  9. tests/unit/test_domain_service.py
  10. tests/unit/test_rbac.py
  11. tests/unit/test_review_engine.py
  12. tests/unit/test_security_dependencies.py
  13. tests/unit/test_workflow_endpoints.py
  14. tests/unit/test_workflow_engine.py
  15. tests/unit/test_workflow_engine_core.py
  16. tests/unit/test_workflow_engine_extended.py
  17. tests/unit/core/test_proactive_scheduler.py

**Approach**:

  1. Fix syntax errors (test_coverage_analyzer.py line 141)
  2. Resolve import dependencies
  3. Fix mock configuration issues
  4. Test each file individually
  5. Verify full test suite runs

**Success Criteria**:

  • All 17 files collect without errors
  • Tests run successfully (passing or failing)
  • Coverage measurement accurate

1.2 Establish Coverage Baseline (2 hours)

**Already Complete** (Phase 65B-01, 66B-01)

  • Frontend: 19.01% baseline ✅
  • Backend: 12.16% baseline ✅
  • HTML reports generated ✅

---

Phase 2: Critical Brain Systems (Week 1, Days 3-5)

Goal

Achieve 90% coverage for core brain systems (graduation, episodes, governance).

2.1 Episode Service (Target: 85-90% coverage)

**Current**: ~0% coverage

**File**: core/episode_service.py (1,570 lines)

**Tests Created**: 19 tests (570 lines) - All failing due to mocks

**File**: tests/core/test_episode_service_comprehensive.py

**Required Fixes** (3-4 hours):

  1. Fix mock configuration for:
  • SQLAlchemy Session mocking
  • LanceDB service mocking
  • Embedding service mocking
  • CanvasAudit trail mocking
  1. Update import statements (EpisodeFeedback from models)
  2. Fix database query mocks
  3. Test and verify all 19 tests pass

**Test Coverage Areas**:

  • Episode creation from executions (3 tests)
  • Graduation readiness calculation (4 tests)
  • Feedback handling (3 tests)
  • Episode retrieval (3 tests)
  • Canvas actions (2 tests)
  • Archival to LanceDB (1 test)
  • Skill performance stats (2 tests)
  • Edge cases (3 tests)

**Estimated Time**: 6-8 hours total

  • Fix mocks: 3-4 hours
  • Add additional tests: 2-3 hours
  • Verify and document: 1 hour

2.2 Graduation Exam Service (Target: 85-90% coverage)

**Current**: ~0% coverage

**File**: core/graduation_exam.py (estimated 800-1000 lines)

**Tests to Create** (20-25 tests):

  1. Exam execution flow (5 tests)
  • Execute exam for agent
  • Multi-stage exam validation
  • Edge case handling
  1. Readiness calculation (5 tests)
  • Calculate readiness metrics
  • Threshold validation
  • Edge cases (insufficient data, boundary conditions)
  1. Level progression (5 tests)
  • Student → Intern promotion
  • Intern → Supervised promotion
  • Supervised → Autonomous promotion
  • Failed progression attempts
  1. Edge cases (5-10 tests)
  • Agent not found
  • Insufficient episodes
  • Concurrent exam attempts
  • Edge case scenarios

**Estimated Time**: 8-10 hours

2.3 Agent Governance Service (Target: 85-90% coverage)

**Current**: ~20% coverage

**File**: core/agent_governance_service.py

**Tests to Create** (15-20 tests):

  1. Permission checks (5 tests)
  • CanPerformAction for all maturity levels
  • Capability validation
  • Resource access checks
  1. Maturity validation (4 tests)
  • Validate maturity requirements
  • Complex action validation
  • Governance checks
  1. Audit logging (3 tests)
  • Action logging
  • Episode linking
  • Audit trail retrieval
  1. Edge cases (3-6 tests)
  • Invalid maturity levels
  • Missing permissions
  • Concurrent requests

**Estimated Time**: 6-8 hours

2.4 Frontend Brain Systems (Target: 85-90% coverage)

**Files**:

  • src/lib/ai/cognitive-architecture.ts (~20% → 90%)
  • src/lib/ai/learning-adaptation-engine.ts (~15% → 90%)
  • src/lib/ai/reasoning-engine.ts (~20% → 90%)
  • src/lib/ai/agent-governance.ts (~25% → 90%)

**Tests to Create**: 30-40 tests total

**Estimated Time**: 8-12 hours

---

Phase 3: Critical API Routes (Week 2, Days 1-3)

Goal

Achieve 85-90% coverage for user-facing API endpoints.

3.1 Graduation Routes (Target: 85-90% coverage)

**File**: api/routes/graduation_routes.py

**Tests to Create** (15-20 tests):

  1. GET /graduation/agents/{id}/readiness (5 tests)
  2. POST /graduation/agents/{id}/exam (5 tests)
  3. POST /graduation/agents/{id}/promote (3 tests)
  4. POST /graduation/agents/{id}/demote (3 tests)
  5. GET /graduation/agents/{id}/history (2 tests)
  6. GET /graduation/edge-cases (2 tests)
  7. POST /graduation/episodes/{id}/feedback (2 tests)

**Estimated Time**: 6-8 hours

3.2 Canvas Skill Routes (Target: 85-90% coverage)

**File**: api/routes/canvas_skill_routes.py

**Tests to Create** (12-15 tests):

  1. POST /canvas-skills/create (3 tests)
  2. POST /canvas-skills/install (3 tests)
  3. POST /canvas-skills/marketplace/permission (2 tests)
  4. POST /canvas-skills/marketplace/submit (2 tests)
  5. GET /canvas-marketplace/components (2 tests)
  6. POST /canvas-marketplace/components/install (2 tests)
  7. POST /canvas-marketplace/components/{id}/rate (2 tests)

**Estimated Time**: 5-7 hours

3.3 Availability Routes (Target: 85-90% coverage)

**File**: api/routes/availability_routes.py

**Tests to Create** (10-12 tests):

  1. Presence tracking (3 tests)
  2. Supervisor assignment (3 tests)
  3. Queue management (2 tests)
  4. Edge cases (2-4 tests)

**Estimated Time**: 4-6 hours

3.4 Proposal Routes (Target: 85-90% coverage)

**File**: api/routes/proposal_routes.py

**Tests to Create** (10-12 tests):

  1. Proposal creation (3 tests)
  2. Proposal approval (3 tests)
  3. Proposal rejection (2 tests)
  4. Proposal listing (2 tests)
  5. Edge cases (2-4 tests)

**Estimated Time**: 4-6 hours

---

Phase 4: Integration & Frontend Coverage (Week 2, Days 4-5)

Goal

Bring frontend coverage to 80% and complete integration tests.

4.1 Frontend AI Systems (Target: 85-90% coverage)

**Files**:

  • src/lib/ai/episodic-memory.ts (add tests)
  • src/lib/ai/graduation-exam.ts (improve coverage)
  • src/lib/ai/agent-runner.ts (add tests)

**Estimated Time**: 8-10 hours

4.2 Integration Clients (Target: 85% coverage)

**Files**:

  • src/lib/integrations/slack.ts
  • src/lib/integrations/notion.ts
  • src/lib/integrations/github.ts

**Estimated Time**: 6-8 hours

4.3 Canvas Components (Target: 80% coverage)

**Files**:

  • src/components/canvas/CanvasEditor.tsx
  • src/components/canvas/PropertyPanel.tsx
  • src/components/canvas/ComponentLibrary.tsx

**Estimated Time**: 6-8 hours

---

Phase 5: Module Completion (Week 3, Days 1-2)

Goal

Bring all remaining modules to 80% minimum coverage.

5.1 Backend Models & Schemas (Target: 85% coverage)

**Files**:

  • core/models.py (validation, relationships)
  • api/schemas/*.py (request/response validation)

**Estimated Time**: 4-6 hours

5.2 Utility Functions (Target: 80% coverage)

**Files**:

  • src/lib/utils/*.ts
  • core/utils/*.py

**Estimated Time**: 3-4 hours

5.3 Service Layer Completion (Target: 80% coverage)

**Files**: Remaining services below 80%

**Estimated Time**: 4-6 hours

---

Success Metrics

Coverage Targets

CategoryCurrentTargetGap
Frontend Coverage19.01%80%-60.99%
Backend Coverage12.16%80%-67.84%
**Combined Coverage****~20%****80%****-60%**
Brain Systems15-30%90%-60-75%
Critical APIs10-30%85%-55-75%

Test Count Targets

TypeCurrentTargetNew Tests Needed
Frontend Tests3,6065,000+~1,400
Backend Tests2,7864,000+~1,200
**Total****6,392****9,000+****~2,600**

---

Execution Plan

Sprint Structure (3 Weeks)

**Week 1**: Infrastructure + Brain Systems

  • Days 1-2: Fix collection errors
  • Days 3-5: Episode, Graduation Exam, Governance services

**Week 2**: Critical APIs + Frontend

  • Days 1-3: API routes tests
  • Days 4-5: Frontend AI systems, integrations

**Week 3**: Completion & Validation

  • Days 1-2: Models, utilities, completion
  • Days 3-5: Final verification, coverage reports

Daily Routine

**Morning** (4 hours):

  • Add new tests
  • Fix failing tests
  • Run coverage reports

**Afternoon** (4 hours):

  • Integrate tests
  • Fix mock issues
  • Document progress

**End of Day**:

  • Coverage report
  • Test count update
  • Blocker identification

---

Risk Mitigation

Risk 1: Mock Complexity (HIGH)

**Risk**: Unit tests require complex mocks that are brittle

**Mitigation**:

  • Start with integration tests (real database)
  • Use factory functions for mock creation
  • Document mock patterns
  • Create reusable test fixtures

Risk 2: Time Estimation Accuracy (MEDIUM)

**Risk**: 2-3 weeks may be optimistic

**Mitigation**:

  • Weekly reassessment
  • Focus on highest-value modules first
  • Defer low-priority work if needed
  • Document partial progress

Risk 3: Test Maintenance (MEDIUM)

**Risk**: Tests become brittle and break frequently

**Mitigation**:

  • Use integration tests where possible
  • Mock external dependencies consistently
  • Create stable test data
  • Regular test maintenance in sprints

---

Success Criteria

Phase Completion Checklist

  • [ ] All 17 collection errors fixed
  • [ ] Episode Service at 85-90% coverage
  • [ ] Graduation Exam Service at 85-90% coverage
  • [ ] Agent Governance at 85-90% coverage
  • [ ] Frontend brain systems at 85-90% coverage
  • [ ] Critical API routes at 85-90% coverage
  • [ ] Frontend coverage ≥80%
  • [ ] Backend coverage ≥80%
  • [ ] Coverage report generated and documented
  • [ ] Test count ≥9,000

Final Verification

# Frontend
npm run test:coverage

# Backend
pytest --cov=core --cov=api --cov-report=html --cov-report=term

# Verify thresholds
# Frontend: 80%+
# Backend: 80%+

---

Next Steps

  1. **Schedule Sprint**: Book 2-3 weeks for dedicated testing sprint
  2. **Team Assignment**: 1-2 developers focused on testing
  3. **Daily Standups**: Track progress, unblock issues
  4. **Weekly Demos**: Show coverage improvements
  5. **Final Validation**: Comprehensive coverage report

---

Conclusion

Achieving 80% test coverage is a **significant undertaking** requiring **80-120 hours** of focused work. This roadmap provides a **systematic approach** to reach that target through **5 phases** over **3 weeks**.

**Key Success Factors**:

  • Start with infrastructure fixes (unblock existing tests)
  • Focus on high-value modules first (brain systems, critical APIs)
  • Use integration tests to reduce mock complexity
  • Track progress daily with coverage reports
  • Accept partial progress if time runs out

**Expected Outcome**:

  • Production-ready quality baseline
  • Confidence in core system functionality
  • Reduced bug rate in production
  • Faster feature development with test safety net

---

**Document Version**: 1.0

**Last Updated**: 2026-02-22

**Owner**: Testing Team

**Review Date**: End of Week 1 of sprint